{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NumPy under MinPy, with GPU\n",
    "\n",
    "This part of tutorial is also available in step-by-step notebook version on [github](https://github.com/dmlc/minpy/blob/master/examples/tutorials/numpy_under_minpy.ipynb). Please try it out!\n",
    "\n",
    "## Basic NDArray Operation\n",
    "\n",
    "MinPy has the same syntax as NumPy, which is the language of choice for numerical computing, and in particular deep learning. The popular [Stanford course cs231n](http://cs231n.stanford.edu/syllabus.html) \n",
    "uses NumPy as its main coursework. To use NumPy under MinPy, you only need to replace `import numpy as np` with `import minpy.numpy as np` at the header of your NumPy program. if you are not familiar with NumPy, you may want to look up [NumPy Quickstart Tutorial](https://docs.scipy.org/doc/numpy-dev/user/quickstart.html) for more details.\n",
    "\n",
    "Using NumPy under MinPy has two simple but important reasons, one for productivity and another for performance: 1) Auto-differentiation, and 2) GPU/CPU co-execution. We will discuss them in this tutorial.\n",
    "\n",
    "But first, let us review some of the most common usages of NumPy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Array Creation \n",
    "An array can be created in multiple ways. For example, we can create an array from a regular Python list or tuple by using the `array` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import minpy.numpy as np\n",
    "\n",
    "a = np.array([1,2,3])  # create a 1-dimensional array with a python list\n",
    "b = np.array([[1,2,3], [2,3,4]])  # create a 2-dimensional array with a nested python list "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are some useful ways to create arrays with initial placeholder content. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "a = np.zeros((2,3))    # create a 2-dimensional array full of zeros with shape (2,3)  \n",
    "b = np.ones((2,3))     # create a same shape array full of ones\n",
    "c = np.full((2,3), 7)  # create a same shape array with all elements set to 7\n",
    "d = np.empty((2,3))    # create a same shape whose initial content is random and depends on the state of the memory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basic Operations\n",
    "Arithmetic operators on arrays apply *elementwise*, with a new array holding result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-2. -2. -2.]\n",
      " [-2. -2. -2.]]\n",
      "[[-0.7568025 -0.7568025]\n",
      " [-0.7568025 -0.7568025]\n",
      " [-0.7568025 -0.7568025]]\n",
      "[[ 2.  2.  2.]\n",
      " [ 2.  2.  2.]]\n"
     ]
    }
   ],
   "source": [
    "a = np.ones((2,3))\n",
    "b = np.ones((2,3))\n",
    "c = a + b  # elementwise plus\n",
    "d = - c    # elementwise minus\n",
    "print(d)\n",
    "e = np.sin(c**2).T  # elementwise pow and sin, and then transpose\n",
    "print(e)\n",
    "f = np.maximum(a, c)  # elementwise max\n",
    "print(f)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Indexing and Slicing\n",
    "The slice operator `[]` applies on axis 0. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.  1.]\n",
      " [ 2.  3.]\n",
      " [ 4.  5.]]\n",
      "[[ 0.  1.]\n",
      " [-1. -1.]\n",
      " [ 4.  5.]]\n"
     ]
    }
   ],
   "source": [
    "a = np.arange(6)\n",
    "a = np.reshape(a, (3,2))\n",
    "print(a[:])\n",
    "# assign -1 to the 2nd row\n",
    "a[1:2] = -1 \n",
    "print(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also slice a particular axis with the method `slice_axis`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.]\n",
      " [-1.]\n",
      " [ 5.]]\n"
     ]
    }
   ],
   "source": [
    "# slice out the 2nd column\n",
    "d = np.slice_axis(a, axis=1, begin=1, end=2)\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## AutoGrad Feature\n",
    "\n",
    "If you work in a policy mode called `NumpyOnlyPolicy` (refer [here](https://minpy.readthedocs.io/en/latest/feature/policy.html) for more details), MinPy is almost compatible with the most of NumPy usages. But what makes MinPy awesome is that it give you the power of autograd, saving you from writing the most tedious and error prone part of deep net implementation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "90\n",
      "43.0\n"
     ]
    }
   ],
   "source": [
    "from minpy.core import grad\n",
    "\n",
    "# define a function: f(x) = 5*x^2 + 3*x - 2\n",
    "def foo(x):\n",
    "    return 5*(x**2) + 3*x - 2\n",
    "\n",
    "# f(4) = 90\n",
    "print(foo(4))\n",
    "\n",
    "# get the derivative function by `grad`: f'(x) = 10*x + 3\n",
    "d_foo = grad(foo)\n",
    "\n",
    "# f'(4) = 43.0\n",
    "print(d_foo(4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "More details about this part can be found in [Autograd Tutorial](http://minpy.readthedocs.io/en/latest/tutorial/autograd_tutorial.html).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## GPU Support\n",
    "\n",
    "But we do not stop here, we want MinPy not only friendly to use, but also fast. To this end, MinPy leverages GPU's parallel computing ability. The code below shows our GPU support and a set of API to make you freely to change the runnning context (i.e. to run on CPU or GPU). You can refer to [Select Context for MXNet](http://minpy.readthedocs.io/en/latest/feature/context.html) for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "run on cpu: 0.100039 s/iter\n",
      "run on gpu: 0.000422 s/iter\n"
     ]
    }
   ],
   "source": [
    "import minpy.numpy as np\n",
    "import minpy.numpy.random as random\n",
    "from minpy.context import cpu, gpu\n",
    "import time\n",
    "\n",
    "n = 100\n",
    "\n",
    "with cpu():\n",
    "    x_cpu = random.rand(1024, 1024) - 0.5\n",
    "    y_cpu = random.rand(1024, 1024) - 0.5\n",
    "\n",
    "    # dry run\n",
    "    for i in xrange(10):\n",
    "        z_cpu = np.dot(x_cpu, y_cpu)\n",
    "    z_cpu.asnumpy()\n",
    "\n",
    "    # real run\n",
    "    t0 = time.time()\n",
    "    for i in xrange(n):\n",
    "        z_cpu = np.dot(x_cpu, y_cpu)\n",
    "    z_cpu.asnumpy()\n",
    "    t1 = time.time()\n",
    "\n",
    "with gpu(0):\n",
    "    x_gpu0 = random.rand(1024, 1024) - 0.5\n",
    "    y_gpu0 = random.rand(1024, 1024) - 0.5\n",
    "\n",
    "    # dry run\n",
    "    for i in xrange(10):\n",
    "        z_gpu0 = np.dot(x_gpu0, y_gpu0)\n",
    "    z_gpu0.asnumpy()\n",
    "\n",
    "    # real run\n",
    "    t2 = time.time()\n",
    "    for i in xrange(n):\n",
    "        z_gpu0 = np.dot(x_gpu0, y_gpu0)\n",
    "    z_gpu0.asnumpy()\n",
    "    t3 = time.time()\n",
    "\n",
    "print(\"run on cpu: %.6f s/iter\" % ((t1 - t0) / n))\n",
    "print(\"run on gpu: %.6f s/iter\" % ((t3 - t2) / n))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `asnumpy()` call is somewhat mysterious, implying `z_cpu` is not NumPy's `ndarray` type. Indeed this is true. For fast execution, MXNet maintains its own datastrcutre `NDArray`. This calls re-synced `z_cpu` into NumPy array.\n",
    "\n",
    "As you can see, there is a gap between the speeds of matrix multiplication in CPU and GPU. That's why we set default policy mode as `PreferMXNetPolicy`, which means MinPy will dispatch the operator to MXNet as much as possible for you, and achieve transparent fallback while there is no MXNet implementation. MXNet operations run on GPU, whereas the fallbacks run on CPU.\n",
    "\n",
    "See [Transparent Fallback](https://minpy.readthedocs.io/en/latest/tutorial/transparent_fallback.html) for more details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Something You Need to Know\n",
    "\n",
    "With [Transparent Fallback](http://minpy.readthedocs.io/en/latest/tutorial/transparent_fallback.html), we hope to transparently upgrade the running speed without your changing a line of code. This can be done by expanding the MXNet GPU operators.\n",
    "\n",
    "However, there are some important [pitfalls](http://minpy.readthedocs.io/en/latest/feature/limitation.html) you should know when you try to use MinPy, we strongly suggest that you should read it next."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
